Touch-based devices, such as smartphones, tablets, and laptops, have become a ubiquitous aspect of daily life. For example, it is now common that a person will organize and conduct social interactions through applications on a smartphone or tablet. Further, businesses often rely on touch-based devices to communicate with employees, monitor jobs, and review project data, among any number of other uses. Touch-based devices typically run touch-based operating systems (such as Android or iOS), which inherently rely on touch-based inputs to control interaction with the operating systems. As useful and valuable as these devices are, they all have the same limitation; at a primary level, they require a user to interact manually directly with the device.
For example, touch-based operating systems now primarily rely on virtual keyboards for receiving textual input. Virtual keyboards often have relatively small buttons where words must be spelled out one letter at a time, which can be time-consuming and impractical for inputting messages of even moderate length. Some touch-based operating systems provide speech recognition to translate text into words, however, such speech recognition often requires an Internet connection, which may not always be available. Further, even if speech recognition is available, it is typically limited to textual input, and touch-based operating systems provide limited mechanisms for navigating user interfaces within applications and navigating between multiple applications. For example, navigating a multipage document in touch-based operating systems generally requires the user to touch the screen at the scroll bar to navigate horizontally, or “swipe” the screen to scroll.
However, users may desire (or need) to use these devices during periods where manual touch-based interaction is infeasible, difficult, or impossible. For example, many individuals may not have the ability to interact physically with touch-based devices, yet need or desire to do so. Additionally, even when a user can physically interact with a touch-based device, environmental restrictions may restrict manual interactions with devices in a safe or comfortable manner. Further, it may be desirable to access functionalities of a touch-based device while a user is engaged in a task that requires the user's hands.
Previous attempts to solve these problems often relied on highly specialized, purpose-built devices or processes that provided cumbersome and inflexible user experiences. Further, previous solutions required modification of source code from existing applications designed for touch-based operating systems, such that the applications were compatible with hands-free input sources. It would also be impractical to modify every touch-based application to facilitate a touch-free solution. Accordingly, existing solutions were only compatible with a few select touch-based applications. Additionally, prior solutions have failed to enable interaction with the touch-based operating systems themselves, and instead rely on their own operating system. As a result, prior solutions generally required a user to have to learn a brand new operating environment, rather than enable interaction with existing and heavily-adopted touch-based operating systems.